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The baryon acoustic oscillation feature in the clustering of galaxies 
or quasars provides a ‘standard ruler’ for distance measurements in 
cosmology. In this work, we report a 2-30 signal of the baryon acoustic 


oscillation dip feature in the galaxy density-ellipticity cross-correlation 
functions using the spectroscopic sample of the Baryon Oscillation 
Spectroscopic Survey CMASS, combined with the deep Dark Energy 
Spectroscopic Instrument Legacy Imaging Surveys for precise galaxy shape 
measurements. We measure the galaxy-ellipticity correlation functions and 
model them using the linear alignment model. We constrain the distance 
D,/rqto redshift 0.57 to a precision of 3-5%, depending on the details of 
modelling. The galaxy-ellipticity measurement reduces the uncertainty 

of distance measurement by ~10% on top of that derived from the galaxy- 
galaxy correlation. More importantly, for future large and deep galaxy 
surveys, the independent galaxy-ellipticity measurements can help sort out 
the systematics in the baryon acoustic oscillation studies. 


Measuring the expansion history of the Universe is one of the key goals in 
cosmology. The best constraints now are those from the measurements 
of the distance-redshift relation over a wide range of redshifts’. The 
baryon acoustic oscillation (BAO) feature in the clustering of galaxies is 
a standard ruler for robust distance measurements~*. BAOs arise from 
tight coupling of photons and baryons in the early Universe. Sound waves 
travel through this medium and give rise to a characteristic scale in the 
density perturbations, corresponding to the propagation distance of the 
waves before the recombination. With large galaxy surveys, using BAOs, 
distances have been measured toa per cent level at various redshifts* *. 

Inthe aforementioned studies, only the spatial distributions of gal- 
axies are used, and the shapes and orientations of galaxies are ignored. 
The intrinsic alignment (IA) of galaxies is usually treated as a contami- 
nant in weak-lensing analysis”°. However, as we will demonstrate using 
actual observations in this work, IA is actually a promising cosmological 
probe and contains valuable information. The galaxy-ellipticity (GI) 
and ellipticity—ellipticity (II) IAs were first detected in luminous red 


galaxies"”’, Studies showed that the IA of galaxies can be related to 
the gravitational tidal field using the linear alignment (LA) model“ +6. 
According to the LA model, a BAO feature in both Gl and II correlations 
shows up as a dip, rather than a peak’”’*” as seen in the galaxy-galaxy 
(GG) correlations. Furthermore, the entire two-dimensional and ani- 
sotropic pattern of GI and II correlations may provide information 
additional to that of the BAOs. The results were tested and confirmedin 
N-body simulations”. Taruya and Okumura” also performed a forecast 
of cosmological constraints for IA statistics using the LA model, and 
found that IA can provide a similar level of constraints on cosmological 
parameters to the galaxy spatial distributions. 

Inthis Article, we report ameasurement of the BAO feature using 
IA statistics, namely GI, and confirm that IA can provide additional 
information to GG correlations. In addition to reducing statistical 
uncertainties of the distance measurements, GI can also provide a 
test of systematics when compared withthe BAO measurements from 
GG. Details of this analysis, including the datasets, measurements 
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Fig. 1 | Measurements and modelling of GI and GG correlation functions. 

a, Pre-reconstruction GI correlations. b, Post-reconstruction GI correlations. 

c, Post-reconstruction GG correlations. d, Post-reconstruction combined 
modelling (GI multiplied by 4 for better illustration). Points and error bars show 
the mean and s.e.m. of clustering measurements. Errors are from the diagonal 
elements of the jackknife covariance matrices estimated using 400 subsamples. 
Lines and shading are the best-fit models and 68% confidence-level regions 
derived from the marginalized posterior distributions. 


and modelling, are presented in Methods. Our fiducial results, as 
shown in the main text, are derived using data vectors in the range of 
50-200 h” Mpc with a polynomial marginalized over. This range is cho- 
sen to ensure that the polynomial captures the entire shape and to avoid 
the turnover around 20 h Mpc. For completeness, we also show results 
with other choices of the fitting range and with or without marginalizing 
over the polynomial in Supplementary Figs. 4 and 5 and Supplementary 
Table 1. We observe a 2-30 BAO signal in all of these variations. 

The measurements and modelling of the isotropic Gl and GG cor- 
relation functions are shown in Fig. 1. The GI measurements show 
an apparent dip around the BAO scale at -100 A Mpc from both the 
pre- (panel a) and post-reconstructed samples (panel b). The fitting 
results are reasonable on all scales, namely, the reduced x” is 0.97 for 
pre-reconstruction and 0.87 for post-reconstruction. To show the 
significance of the BAO detection, we display the Ay? = x” — Xmin SUF- 
faces in Fig. 2 (see also Supplementary Fig. 1), where Xmin is the x’ for 
the best-fitting model. We compare Ay’ for the no-wiggle model with 
the BAO model and find a 30 detection of the BAO feature in both the 
pre- and post-reconstructed samples. 

The constraint on a, which represents the deviation from the 
fiducial cosmology (Methods), is 1.050*9 03? and 1.057+0-035 using the 
pre- and post-reconstructed samples, respectively. Both results are in 
good agreement (within 20) with the fiducial Planck18 (ref. 22) results 
(TT, TE, EE + lowE + lensing + BAO), which assumes aA cold dark matter 
model (where A is the cosmological constant) cosmology. We find that 
the constraint is not improved after reconstruction for the GI correla- 
tions, which may be due to the fact that we only reconstruct the density 
field and keep the shape field unchanged. The result may be further 
improved in principle ifthe shape field is also reconstructed, whichis 
left for a future study. 

The post-reconstructed GG (panel c) and combined (panel d) 
measurements and modelling are also shown in Fig. 1. The GG correla- 
tion alone measures a to be 0.986703 and the combined GG + GI 
derives the constraint to be 0.997700. demonstrating that the GI 
measurement gives rise to a -10% improvement in terms of the uncer- 
tainty on a. More importantly, as we mentioned above, the next- 
generation surveys can tighten the GG BAO constraints by a factor of 
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Fig. 2 | Plot of Ay’ versus a for the pre- (left) and post- (right) reconstruction GI 
correlations. Orange lines show the Ay’ for non-BAO models and blue lines those 
for BAO models. 
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Fig. 3 | Constraints of D,/r, from GG and GI correlation functions with 

Nx = 400. A combined post-reconstructed GG + Gl constraint is also provided. 
The central values are the medians, and the error bars are the 16th and 84th 
percentiles. The vertical orange line shows the fiducial Planck18 results. 


2-3 (refs. 1,23), and systematic errors will become more and more 
important. Ifthe GI measurements can be improved to the same level, 
comparisons between sub-per-cent (<0.5%) GG measurements and 1% 
GI measurements can provide a check of the systematic bias in the 
measurements. Our GG results are consistent with the results reported 
by the Baryon Oscillation Spectroscopic Survey (BOSS) team‘ within 
the 68% confidence level uncertainty, although the numbers are slightly 
different due to a few effects, including fitting ranges, details of radial 
bins and error estimations. 

In Fig.3, we convert the constraints on ato distance D,/r;measured 
at redshift z = 0.57, with the fiducial values Dyfa(0.57) = 2,056.58 Mpc, 
Fania = 147.21 Mpc and Dyfia/fa iq = 13.97 for Planck18 (ref. 22). The quan- 
tity D,/r, is measured to be 14.67+°42 , 14.77*°49, 13.77+918 and 


oe -0.40’ -0.50" -0.19 
13.91*°-” using the pre-reconstructed GI, post-reconstructed GI, GG 


and Gi? GG, respectively. All these results are consistent with Planck18 
within the 2olevel. 

In this work, we obtain a 2-30 measurement of the BAO dip feature 
in Gl correlations, although the constraints on the distance from Glare 
only around one-third of those from GG, much weaker than predicted 
by Taruya and Okumura” using the LA model. The reason may be that 
the galaxy-halo misalignment” can reduce the IA signals and weaken 
the BAO constraints, which may not be considered appropriately by 
Taruya and Okumura”. According to Okumura and Jing™, on taking 
into account the misalignment the GI signals can be reduced by two- to 
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threefold, which is consistent with our results. Moreover, since realistic 
mock catalogues for galaxy shapes are unavailable as yet, the covari- 
ance matrices in this study are estimated using the jackknife resam- 
pling method. Employing more reliable error estimation techniques 
could potentially improve the accuracy of the results, and is left fora 
future study. Nevertheless, the results are already promising. With the 
next-generation spectroscopic and photometric surveys including 
the Dark Energy Spectroscopic Instrument (DESI)” and the Legacy 
Survey of Space and Time”, we will have larger galaxy samples and bet- 
ter shape measurements. We expect that the IA statistics can provide 
much tighter constraints on cosmology from BAO and other probes”. 


Methods 

Statistics of the IA 

The shape of galaxies can be characterized by a two-component 
ellipticity, which is defined as follows: 


1-7 : 
Ya,» = ig ee ; (1) 


where g represents the minor-to-major axial ratio of the projected 
shape, and 0 denotes the angle between the major axis projected onto 
the celestial sphere and the projected separation vector pointing 
towards a specific object. 

The Gl correlations, denoted as the cross-correlation functions 
between density and ellipticity fields, can be expressed as™”® 


Sgil) = (M + gD + 6g (Xo) ]Vi(%2)). (2) 


Here, 5, is the overdensity of galaxies, r = x, - x, and i = {+, x}. 

In this work, we focus on the GI correlation €,, since the signal of 
¿x vanishes due to parity considerations. It is worth noting that the IA 
statistics exhibit anisotropy even in real space due to the utilization 
of projected shapes of galaxies, and the presence of redshift space 
distortion” can introduce additional anisotropies in g(r). Therefore, 
we define the multipole moments of the correlation functions as”? 


1 
mtl f du XAB). 6) 
=i 


Xe) = 


Here, X represents one of the correlation functions, % denotes the 
Legendre polynomials and u corresponds to the directional cosine 
between rand the line-of-sight direction. 


TheLA model 

On large scales, the LA model is frequently employed in studies 
of IAs'*>8, This model assumes a linear relationship between the 
ellipticity fields of galaxies and halos and the gravitational tidal field. 


G 
V4.2) 00 = -zag (Vi — Vj, 2VxVy) Yp), (4) 


where W, represents the gravitational potential, G denotes the gravi- 
tational constant and C, characterizes the strength of IA. Although the 
observed ellipticity field is density weighted, namely [1 + 6,(x)]7...9(x), 
the term 6,(X)¥.9(x) is subdominant on large scales" because 
6,(x) < 1, and it can be neglected at the BAO scale. In the Fourier space, 
equation (4) can be expressed as 

(k — Ks 2k,ky) 


Yax (K) = -Č È 8k). (5) 


Here, Č (a) = œ Gp(a)/ D(a), where f represents the mean mass density 
of the Universe, D(a) x D(a)/a, and D(a) corresponds to the linear 
growth factor, with a denoting the scale factor. 


Then, €,.(r) can be represented by the matter power spectrum 
PS 


K dk 
212 


P5g(k Yo(kr), (6) 


E(t) = Cbg — 42) [ 
0 


where b, is the linear galaxy bias and j, is the second-order spherical 
Bessel function. 

The redshift space distortion effect” can also be considered in 
é,.(r) at large scales’®. However, in this work, we do not consider the 
redshift space distortion effect and only focus onthe monopole com- 
ponent of ¢,.(r) given the sensitivity of current data. 


© Kdk 


Ege) = FCaby | LS Pooll yatkr). O 


We plan to measure the entire two-dimensional €,,(r) with future large 
galaxy surveys, which may contain much more information. To test the 
LA model, Okumura et al.” measured the IA statistics in N-body simula- 
tions and found that the results agree well with the predictions from 
the LA model on large scales. Thus, it is reasonable to use the above 
formula of + o(r) for BAO studies. 


Fitting the BAO scale 
We fit the BAO features in GG correlations following the SDSS-III BOSS 
DR12 analysis”. 

To model the BAO features in GI correlations, we adopt a meth- 
odology similar to that used in GG studies”. In spherically averaged 
two-point measurements, the BAO position is fixed by the sound hori- 
zon at the baryon-drag epoch r; and provides a measurement of* 


D2) = [cz + ORON (8) 


where D, (z) is the angular diameter distance and H(z) is the Hubble 
parameter. The correlation functions are measured under an assumed 
fiducial cosmological model to convert angles and redshifts into dis- 
tances. The deviation of the fiducial cosmology from the true one 
can be measured by comparing the BAO scale in clustering measure- 
ments with its position in a template constructed using the fiducial 
cosmology. The deviation is characterized by 


DyY@)a,fia 
Dy fia ra’ 


(9) 


a= 


where the subscripts ‘fid’ denote the quantities from the fiducial 
cosmology. 

The template of €,,.. is generated using the linear power spectrum, 
Pin from the CLASS code”. In GG BAO peak fitting, a linear power spec- 
trum with damped BAOis usually used to account for the nonlinear effect, 


Pink) 
Paw(k) 


(10) 


Paamp(k) = Paw (k) [1 ( 1) eumesa], 


where P,,, is the fitting formula of the no-wiggle power spectrum’ and 
2,, is the damping scale. In this analysis, we set 2,, = O as our fiducial 
model for GI, and we also show the results with Z as a free parameter 
in Supplementary Table 1. 
Using the template, our model for Gl correlation is given by 
© 2 dk 


Zaz Pin KYi2(aks), 


2112 a) 


§e1,0(S) =B 
0 


where sis the comoving distance in redshift space and B accounts for 
all the factors that only affect the amplitude of the correlation, such 
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as IA strength, galaxy bias and shape responsivity (see equation (16)). 
As in the GG analysis, we add a further polynomial in our model to 


marginalize over the broad band shape: 
a 


a 
od i M 
gr 05) = Se4,0(8) + z7 + 


(12) 


+ a3. 


Thus, with the observed GI correlation §40°°(S) and the covariance 
matrix C, we can assume a likelihood function £ « exp(—x?/2), with 
Nyx — Moin — 2 

Nyx —1 


Os > [& mod _ & obs | CG; Ig mod _ § 05] ; (13) 


ij 


where C ‘is the inverse of C, i,j indicate the data points at different 
radial bins, Nand N, are the total numbers of subsamples and radial 
bins and (Nx - Nyin ~ 2)/(Njx - 1) is the Hartlap correction factor” to 
obtain the unbiased covariance matrix. The covariance matrices are 
estimated using the jackknife resampling from the observation data: 


(14) 


where & is the measurement in the nth subsample at the ith radial bin 
and & is the mean jackknife correlation function at the ith radial bin. 
We use the Markov chain Monte Carlo sampler emcee” to perform a 
maximum-likelihood analysis. Nj, is chosen by gradually increasing it 
until the constraints on aware stable (Supplementary Table 1). 


Sample selection 

We use the data from the BOSS CMASS DR12 sample®*””. The CMASS 
sample covers an effective area of 9,329” and provides spectra for over 
0.8 million galaxies. Galaxies are selected with anumber of magnitude 
and colour cuts to obtain an approximately constant stellar mass. We 
use the CMASSLOWZTOT Large-Scale Structure catalogue in BOSS 
DR12 and adopt a redshift cut of 0.43 < z < 0.70 to select the CMASS 
sample with an effective redshift Ze = 0.57. 

Reconstruction methods can improve the significance of the 
detection of the BAO feature, and reduce the uncertainty in BAO 
scale measurements, by correcting for the density field smooth- 
ing effect associated with large-scale bulk flows***°. We also use the 
post-reconstructed catalogues from BOSS DR12 and we refer for the 
details of the reconstruction methods to their papers®*°. 

To obtain high-quality images for the CMASS galaxies, we 
cross-match them with the DESI Legacy Imaging Surveys DR9 data”! 
(https://www.legacysurvey.org/dr9/files/#survey-dr9-region-speco 
bj-dr16-fits), which cover the full CMASS footprint and contain all 
the CMASS sources. The Legacy Surveys can reach an r-band point 
spread function depth fainter than 23.5 mag, which is two to three 
magnitudes deeper than the SDSS photometry survey used for target 
selection and is more than adequate to study the orientations of the 
massive CMASS galaxies. The Legacy Survey images are processed 
using Tractor”, a forward-modelling approach to perform source 
extraction on pixel-level data. We use shape _el and shape _e2 in the 
Legacy Surveys DR9 catalogues (https://www.legacysurvey.org/dr9/ 
catalogs/#ellipticities) as the shape measurements for each CMASS gal- 
axy. These two quantities are then converted to the ellipticity defined 
in equation (1). Following Okumura & Jing”, we assume that all the 
galaxies have q = 0, which is equivalent to assuming that a galaxy is a 
line along its major axis. This assumption only affects the amplitude of 
the GI correlations, and the measurements of the position angles are 
more accurate than those of the whole galaxy shapes. 

The whole CMASS sample is used to trace the density field, while for 
the tracers of the ellipticity field we further select galaxies with Sérsic 
index’ n > 2,since only elliptical galaxies show strong shape alignments, 
and q< 0.8 for reliable position angle measurements. In principle, we 


should also exclude satellite galaxies. However, since selecting centralsin 
redshift space is arbitrary and most of the CMASS luminous red galaxies 
are already centrals, we do not consider the central-satellite separation. 
Selections using n and g remove nearly half (from 816,779 to 425,823) 
of the CMASS galaxies. We show the results using the whole sample in 
Supplementary Table 1 and confirm that the morphology and q selec- 
tioncan really improve the measurements and tighten the constraints. 


Estimators 

To estimate the GI correlations, we generate two random samples R, 
and R for the tracers of ellipticity and density fields respectively. Fol- 
lowing Reid et al.’’, redshifts are assigned to randoms to make sure 
that the galaxy and random catalogues have exactly the same redshift 
distribution. We adopt the generalized Landy-Szalay estimator**** 


S,(D-R) 


§e+,0(S) = 


where R R is the normalized random-random pairs. S,D is the sum of 
the + component of ellipticity in all pairs: 


(16) 


where the ellipticity of the jth galaxy in the ellipticity tracers is defined 
relative to the direction to the ith galaxy in the density tracers, and 
R =1—(y?)isthe shape responsivity*®. R = 0.5 under our q = Oassump- 
tion. S,R is calculated in a similar way using the random catalogue. 

We also measure the Gl correlation functions for the reconstructed 
catalogues. The ellipticities of galaxies are assumed unchanged inthe 
reconstruction process. The estimator becomes 


S,E-1) 


§c4,0(S) = RR” (17) 


where E and T represent the reconstructed data and random sample, 
and Rand R, are the original random samples. Inthe above calculations, 
we adopt the Feldman-Kaiser-Peacock weights” (w,,,) and weights for 
correcting the redshift failure (w,,), fibre collisions (w.p) and image sys- 
tematics (w,,,) for the density field tracers”: Wot = WexpWsys(Wep + We 1), 
while no weight is used for the ellipticity field tracers. 


Measurements and modelling 

We measure €,,..(s) for both pre- and post-reconstruction catalogues 
in 50 < s<200 h” Mpc with a bin width of 5 h” Mpc. We calculate their 
covariance matrices using the jackknife resampling with Ny = 400, and 
model the GI correlation functions using equation (11) with a Planck18 
(ref. 22) fiducial cosmology at z= 0.57. In Supplementary Table 1, we 
show that the pre- and post-reconstruction GI results are relatively 
stable if Ny = 400, verifying that Nj. = 400 is a reasonable choice. We 
measure and model the post-reconstruction isotropic GG correlation 
functions with the same radial bins and error estimation schedule 
(Nyx = 400). We also model the GG and GI correlation together with a 
60 x 60 covariance matrix that includes the GG-GI cross-covariance 
to obtain the combined results. 


Data availability 

All data used in this study are publicly available. SDSS-III BOSS DR12 
Large-Scale Structure catalogue: https://data.sdss.org/sas/dr12/boss/ 
Iss/. DESI Legacy Imaging Surveys DR9 catalogue: https://www.lega- 
cysurvey.org/dr9/catalogs/. 
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